{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "%pip install datasets transformers[tf,torch,sentencepiece,vision,optuna,sklearn,onnxruntime]==4.11.3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#hide\n",
    "from utils import *\n",
    "setup_chapter()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this book we will demonstrate how you can run the example from the book in Amazon SageMaker. \n",
    "\n",
    "The SageMaker notebook uses an AWS IAM role to access AWS resources such as Amazon S3 bucket.\n",
    "You created this role during the notebook creation process described in the README.md in SageMaker/README.md.\n",
    "In the AWS IAM service you are able to review the access policy and you can modify it.\n",
    "\n",
    "In the next cell we will check an Amazon S3 bucket exists and create a new one if not. In addition we'll get the SageMaker role and session."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sagemaker.huggingface\n",
    "import sagemaker\n",
    "\n",
    "sess = sagemaker.Session()\n",
    "# sagemaker session bucket -> used for uploading data, models and logs\n",
    "# sagemaker will automatically create this bucket if it not exists\n",
    "sagemaker_session_bucket=None\n",
    "if sagemaker_session_bucket is None and sess is not None:\n",
    "    # set to default bucket if a bucket name is not given\n",
    "    sagemaker_session_bucket = sess.default_bucket()\n",
    "\n",
    "role = sagemaker.get_execution_role()\n",
    "sess = sagemaker.Session(default_bucket=sagemaker_session_bucket)\n",
    "\n",
    "print(f\"sagemaker role arn: {role}\")\n",
    "print(f\"sagemaker bucket: {sess.default_bucket()}\")\n",
    "print(f\"sagemaker session region: {sess.boto_region_name}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we setup a helper function to easily deploy any Hugging Face model as an endpoint on AWS SageMaker.\n",
    "We use the following function to create a HuggingFaceModel Class, where we are going to download the model from the Hugging Face hub. This class would also allow to use a trained model stored on the Amazon S3 bucket. \n",
    "Next, an endpoint will be created and this endpoint will host your model. Based on the model requirements you can choose a specific instance type which are equipped differently in memory, cpu, gpu. There are different inferences available, such as real-time, asynchronous or serverless.\n",
    "If you are not sure which inference works best for your model, you use Amazon SageMaker Inference Recommender.\n",
    "https://docs.aws.amazon.com/sagemaker/latest/dg/deploy-model.html \n",
    "\n",
    "To view all options see the documentation: https://sagemaker.readthedocs.io/en/stable/frameworks/huggingface/index.html \n",
    "\n",
    "Depending on Transformer version, PyTorch/TensorFlow version and Python version, the mapping for the Hugging Face Model Class can be found here: https://huggingface.co/docs/sagemaker/reference#inference-dlc-overview \n",
    "\n",
    "To find the endpoints in the AWS Console navigate to https://console.aws.amazon.com/sagemaker/home#/endpoints \n",
    "\n",
    "Make sure to finish this notebook to delete the endpoint in the end. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.huggingface.model import HuggingFaceModel\n",
    "\n",
    "def setup_endpoint(model_name, task_name):\n",
    "    # Hub Model configuration. <https://huggingface.co/models>\n",
    "    hub = {\n",
    "      'HF_MODEL_ID': model_name, # model_id from hf.co/models\n",
    "      'HF_TASK': task_name # NLP task you want to use for predictions\n",
    "    }\n",
    "\n",
    "    # create Hugging Face Model Class\n",
    "    huggingface_model = HuggingFaceModel(\n",
    "       env=hub, # configuration for loading model from Hub\n",
    "       role=role, # iam role with permissions to create an Endpoint\n",
    "       transformers_version=\"4.17.0\", # transformers version used\n",
    "       pytorch_version=\"1.10.2\", # pytorch version used\n",
    "       py_version=\"py38\" # python version used\n",
    "    )\n",
    "\n",
    "    # deploy model to SageMaker Inference\n",
    "    predictor = huggingface_model.deploy(\n",
    "       initial_instance_count=1, # how many instances used\n",
    "       instance_type=\"ml.m5.xlarge\" # instance type\n",
    "    )\n",
    "    return predictor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Hello Transformers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Encoder-Decoder Framework"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Attention Mechanisms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Transfer Learning in NLP"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hugging Face Transformers: Bridging the Gap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A Tour of Transformer Applications"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "text = \"\"\"Dear Amazon, last week I ordered an Optimus Prime action figure \\\n",
    "from your online store in Germany. Unfortunately, when I opened the package, \\\n",
    "I discovered to my horror that I had been sent an action figure of Megatron \\\n",
    "instead! As a lifelong enemy of the Decepticons, I hope you can understand my \\\n",
    "dilemma. To resolve the issue, I demand an exchange of Megatron for the \\\n",
    "Optimus Prime figure I ordered. Enclosed are copies of my records concerning \\\n",
    "this purchase. I expect to hear from you soon. Sincerely, Bumblebee.\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Text Classification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor = setup_endpoint('distilbert-base-uncased-finetuned-sst-2-english', 'text-classification')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# example request, you always need to define \"inputs\"\n",
    "import pandas as pd\n",
    "\n",
    "# request\n",
    "outputs = predictor.predict({\"inputs\": text})\n",
    "pd.DataFrame(outputs)    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor.delete_endpoint()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Named Entity Recognition"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor = setup_endpoint(\"dbmdz/bert-large-cased-finetuned-conll03-english\", \"ner\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "outputs = predictor.predict({\"inputs\": text, \"parameters\": {\"aggregation_strategy\": \"simple\"}})\n",
    "pd.DataFrame(outputs)    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor.delete_endpoint()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Question Answering "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor = setup_endpoint(\"distilbert-base-cased-distilled-squad\", 'question-answering')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "question = \"What does the customer want?\"\n",
    "\n",
    "outputs = predictor.predict({\"inputs\": {\n",
    "    \"question\": question,\n",
    "    \"context\": text\n",
    "    }\n",
    "})\n",
    "\n",
    "pd.DataFrame([outputs])    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor.delete_endpoint()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Summarization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor = setup_endpoint(\"sshleifer/distilbart-cnn-12-6\", 'summarization')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "outputs = predictor.predict({\"inputs\": text,\n",
    "                             \"parameters\": {\n",
    "                                 \"max_length\":45,\n",
    "                                 \"clean_up_tokenization_spaces\":True\n",
    "                                 }\n",
    "                            })\n",
    "print(outputs[0]['summary_text'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor.delete_endpoint()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Translation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor = setup_endpoint(\"Helsinki-NLP/opus-mt-en-de\", \"translation\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "outputs = predictor.predict({\"inputs\": text,\n",
    "                             \"parameters\": {\n",
    "                                 \"min_length\":100,\n",
    "                                 \"clean_up_tokenization_spaces\":True\n",
    "                                 }\n",
    "                            })\n",
    "print(outputs[0]['translation_text'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor.delete_endpoint()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Text Generation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor = setup_endpoint(\"gpt2\", 'text-generation')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = \"Dear Bumblebee, I am sorry to hear that your order was mixed up.\"\n",
    "prompt = text + \"\\n\\nCustomer service response:\\n\" + response\n",
    "\n",
    "outputs = predictor.predict({\"inputs\": prompt,\n",
    "                             \"parameters\": {\n",
    "                                 \"max_length\":200\n",
    "                                 }\n",
    "                            })\n",
    "print(outputs[0]['generated_text'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor.delete_endpoint()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Hugging Face Ecosystem"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Hugging Face Hub"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hugging Face Tokenizers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hugging Face Datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hugging Face Accelerate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Main Challenges with Transformers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.9.13 64-bit",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  },
  "vscode": {
   "interpreter": {
    "hash": "aee8b7b246df8f9039afb4144a1f6fd8d2ca17a180786b69acc140d282b71a49"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
